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Abstract 

Given a matrix A G K'"^" (n vectors in m dimensions), and a positive integer k < n, we consider the 
problem of selecting k column vectors from A such that the volume of the parallelepiped they define is 
maximum over all possible choices. We prove that there exists S < 1 and c > such that this problem 
(/3 is not approximable within 2~'^* for k — 5n, unless P = NP. 

o 



^ ; 1 Introduction 
> ■ 

' Given a matrix A E M™^", it is of practical importance to obtain the "significant information" contained 

. in A. It becomes especially important to have a compact representation of A when A is large and has 

' low numerical rank, as is typical of modern data. Thus, in a broad sense, we are interested in concise 

representations of matrices. Besides the tremendous practical impact of linear algebraic algorithms designed 



^ ■ to this aim, they also come up in different theoretical forms and paradigms. Specifically, the formalization 



of "significant information" can be done in several ways and to a great extent, it depends on how a matrix 
is interpreted. 

From a conceptual point of view, rather than interpreting a matrix as a block of numbers, we view it 
as a set of vectors (specifically, column vectors) which are indivisible entities. Thus, the formalization of 
. "significant information" is essentially related to finding a subset of columns of the matrix which satisfies 

^ ' some certain spectral conditions or orthogonality requirements. From a purely combinatorial perspective, 

treating vectors as elements of a set, one can also view subset selection in matrices as a generalization of 
the usual subset selection problem where the elements contain little or no information. To give a specific 
example, the well known Set Cover problem asks for a smallest cardinality subset of a set system which 
covers a universal set. Likewise, the problem we are interested in essentially asks for a small number of 
column vectors to "cover" the whole matrix. In this paper, we state a measure of quality for this problem, 
namely the volume, and we prove an exponential inapproximability result for the problem of selecting a 
maximum volume sub-matrix of a matrix. 

Several problems in matrix analysis require to construct a more concise version of a matrix generally 
performed by a re-ordering of the columns jl4] , such that the new smaller matrix is as good a representative 
of the original as possible. One of the criteria that defines the quality of a subset of columns of a matrix 
is how well-conditioned the sub-matrix that they define is. To motivate the discussion, consider the set of 
three vectors 
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which are clearly dependent, and any two of which are a basis. Thus any pair can serve to reconstruct all 
vectors. Suppose we choose ei,u as the basis, then 62 = {l/e)u — (Vi--?/e)ei, and we have a numerical 
instability in this representation as e — 0. Such problems get more severe as the dimensionality of the 
space gets large (curse of dimensionality), and it is natural to ask the representatives to be "as far away 
from each other as possible" . From this simple example, we see that two orthogonal vectors will capture 
more information about a superset of columns than two that have an acute angle between each other. 
Hence, in its generality, this vaguely stated problem can be stated as finding a subset of columns with the 
maximum volume possible or equivalently with the maximum determinant. A similar (but not equivalent) 
problem is to find a subset with the maximum smallest singular value. Indeed, in one of the early works 
studying Rank Revealing QR (RRQR) factorizations [5D] , while discussing different options on how to choose 
a good sub-matrix, it was noted that it turns out that "the selection of the sub-matrix with the maximum 
smallest singular value suggested in |13| can be replaced by the selection of a sub-matrix with maximum 
determinant" , which heuristically proposes to maximize the volume of the sub-matrix instead of using more 
complicated functions. Several algorithms have been designed following this intuition [51 1^ [TBI 124) . 
The optimization problem of finding a maximum volume sub-matrix of a matrix was only recently studied 
by Qivril and Magdon-Ismail: 

Definition 1.1. |^ Given a matrix A E K™>^" of rank at least k, MAX-VOL is the problem of finding a 
sub-matrix C S M™^*^ of A such that the volume of the k dimensional parallelepiped defined by the column 
vectors in C is maximum over all possible choices. 

Theorem 1.2. MAX-VOL is NP-hard. Further, it is NP-hard to approximate to within 2V2/3 -I- e for 
arbitrarily small e > 0. 

Since MAX-VOL is NP-hard, it is natural to ask for an algorithm to approximate the maximum volume. 
The first thing one might try is a simple greedy algorithm for approximating MAX-VOL: 



Algorithm 1 Greedy 

1: 

2: while |C| < fc do 

3: Select the largest norm vector v E A 

4: Remove the projection of v from every element of A 

5: C^CUv 

6: end while 



The analysis of the approximation ratio of this algorithm and a lower bound was also provided in [7]. 
Specifically, let Vol{Gr) be the volume of the column vectors chosen by Greedy and let Vol{Opt) be the 
optimum volume. Then, we have 

Theorem 1.3. Q/ Vol{Gr) > ^ ■ Vol{Opt). 

Theorem 1.4. [7] There exists an instance of MAX-VOL for which Vol{Gr) < ^^{1 - e) ■ Vol{Opt) for 
arbitrarily small e > 0. Furthermore, this instance can explicitly be constructed. 

Note that there is a gap between the proven approximation ratio and the lower bound implied by the 
explicit example. The analysis yielding the ratio 1/fc! is essentially a product of k different mutually exclusive 
analyses related to each step of the algorithm. However, it is not clear whether the overall contribution of 
these different steps to the approximation ratio is actually better than their products. Indeed, the lower 
bound of 1/2*^^^ pertains to such a peculiar construction that we have conjectured a 1/2'^'"^ approximation 
ratio for the greedy algorithm. Hence, in general, proving an exponential inapproximability for this problem 
is an important step towards characterizing its approximability properties. It will show that the greedy 
algorithm is almost the best one can hope for. 

This work takes a first step towards this goal and prove exponential inapproximability for MAX-VOL via 
a gap preserving reduction from the well known Label-Cover problem using the Parallel Repetition Theorem 
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[25] , In doing so, we will establish that the greedy algorithm is asymptotically optimal up to a logarithm in 
the exponent. Specifically, we prove the following theorem: 

Theorem 1.5. There exists S < 1 and c > such that the problem MAX-VOL is not approximahle within 
2-ck ^ _ unless P = NP. 

Our reduction may also be of independent interest which can be used to prove inapproximability results 
for other matrix approximation problems with different objective functions. 

1.1 Preliminaries and Notation 

We introduce some preliminary notation and definitions. Let a matrix A be given in column notation as: 
A = {wi, W2, . . . , Wn}- The volume of A, Vol{A) can be recursively defined as follows: if A contains one 
column, i.e. A — {wi}, then Vol{A) = \\v\\2, where || • is the Euclidean norm. If A has more than one 
column, Vol{A) = \\v — t^{a-{v}){v)\\2 ' VollA — {v}) for any v € A, where 7r^(w) is the projection of v onto 
the space spanned by the column vectors of A. It is well known that 7r(-^_{j,-i,') (u) = AyA'^v, where Ay is the 
matrix whose columns are the vectors in A — {u}, and A^ is the pseudo-inverse of Ay (see for example [14]). 
Using this recursive expression, we have 

n-l 

Vol{S) - Vol{A) = Ikilla • n - ^^^t^^+l\\2 

i=l 

where Ai — {vi ■ ■ ■ Vi} for < i < n — 1. 

We observe a simple fact about the "distance" of a vector to a subspace in the following lemma, which 
will be useful in the final proof. Given two sets of vectors P and Q = {gi, . . . , Qm}, let d{q, P) = \\q — 7rp(g)||2 
denote the distance of g e Q to the space spanned by the vectors in P. 

Lemma 1.6 (Union Lemma). Vol{P U Q) < Vol{P) ■ HLi d{qi,P). 

Proof. We argue by induction on m. For m — 1, Q has one clement and the statement trivially holds. 
Assume that it is true for n = k where Q = {qi, . . . , g^}. Then, for any qk+i 

Vol{P UQU {qk+i}) = Vol{P U Q) ■ d{qk+uP U Q) 
<(,) Vol{PUQ)-d{qk+i,P) 
k 

<ib) Vol{P)-l[d{q,,P)-d{qk+i,P) 

1=1 

k+l 

= Vol{P)-Y[d{q,,P). 

i=l 

(a) follows because d{q, AU B) < d{q, A) for any A, B and (b) follows by the induction hypothesis. □ 

1.2 Related Work 

The concept of volume has been closely related to matrix approximation and mainly studied from a linear 
algebraic perspective. There are a few results revealing the relationship between the volume of a subset of 
columns of a matrix and its approximation. In |10| , the authors introduced volume sampling to find low-rank 
approximation to a matrix where one picks a subset of columns with probability proportional to the volume 
of the simplex they define. In volume sampling, one picks a subset of columns S of size k with probability 



where the summation in the denominator is over all subsets of size k. This sampling provides an almost 
tight low-rank approximation of a matrix in Frobenius norm. Improving this existence result, Deshpande and 
Vempala provided an adaptive randomized algorithm for the low-rank approximation problem, which 
includes a sub-procedure that repetitively chooses a small number of columns by approximating volume 
sampling. This algorithm is essentially a greedy algorithm and can be regarded as a randomized version of 
the greedy algorithm we have analyzed for MAX- VOL [Tj. They show that, if Ps is the probability that this 
algorithm chooses a subset of columns S of size fc, then 



Thus, not only is sampling larger volume columns good, but approximately sampling columns with large 
volume can prove useful for matrix approximation. A natural question is to ask what happens when one 
finds a set of columns with the largest volume (deterministic), which is our problem MAX- VOL. Note that, 
the last expression ([l} is reminiscent of the approximation ratio we have proved for MAX- VOL in [7], but 
its analysis relies on a linear algebraic identity whereas the result in [7] is derived via combinatorial means. 
MAX- VOL and volume sampling seem to be related, but they have different characteristics. MAX- VOL 
is proven to be intractable by using complexity theoretic tools, whereas according to a recent result by 
Deshpande and Rademacher [9j, volume sampling can be exactly implemented in polynomial time. This 
work together with [9] reveals the fact that, although one can exactly sample the columns of a matrix with 
probability proportional to their volumes, identifying a subset with the maximum volume is hard. 

Goreinov and Tyrtyshnikov [15] provided explicit statements of how MAX- VOL, in particular, is related 
to low-rank approximations in the following theorem: 

Theorem 1.7. |15) Suppose that A is an m x n block matrix of the form 



where An is nonsingular, k x k, whose volume is at least fi ^ times the maximum volume among all k x k 
sub-matrices. Then\^ \\A22 — ^2i^n^^i2||oo < l^ik + l)(7k+i{A) . 

This theorem implies that if one has a good approximation to the maximum volume k x k sub-matrix, 
then the rows and columns corresponding to this sub-matrix can be used to obtain a good approximation 
to the entire matrix in the 00-norm. If ak+i{A) is small for some small k, then this yields a low-rank 
approximation to A. [16] also proves a similar result to Theorem 11.71 

Pan [23] unifies the main approaches developed for finding RRQR factorizations by defining the concept 
of local maximum volume and then gives a theorem relating it to the quality of approximation. 

Definition 1.8. Let A e R™^" and C be a sub-matrix of A formed by any k columns of A. Vol{C){^ 0) 
is said to be local ^-maximum volume in A, if fiVol{C) > Vol{C') for any C that is obtained by replacing 
one column of C by a column of A which is not in C. 

Tiieorem 1.9. [23] For a matrix A e K"^", an integer k {1 < k < n) and fi > I, let H e M"""" be a 
permutation matrix such that the first k columns of AW is a local fi-maximum in A. Then, for the QR 
factorization 



We note that, MAX- VOL asks for a stronger property of the set of vectors to be chosen, i.e. it asks for a 
"good" set of vectors in a global sense rather than only requiring local optimality. Obviously, a solution to 
MAX- VOL provides a set of vectors with local maximum volume. 

^ IIBIloo denotes the maximum modulus of the entries of a matrix B. 



Ps<k\-Ps. 



(1) 





we have a,, 



{Rii) > il/^k{n-k) ^2 + l)fXfc(A) andai{R22) < ^k{n - k) fi^ + lak+M). 
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Independently of our work, there are some results in computational geometry which are related to the 
ability to construct large simplices embedded in V-polytopes. Essentially, the problem we consider is a more 
general version of finding a large simplex in a V-polytope, where the vertices of the polytope are the column 
vectors. The results in this area are similar to ours in spirit but using different techniques [T71 [5T1 . The 
most relevant work to ours is that of Koutis 1211 , which shows exponential inapproximability for finding a 
large simplex in a V-polytope. He provides a reduction from set packing using an inapproximability result 
of [19] ■ whereas our reduction is directly from the Label Cover problem. 

2 The Label-Cover Problem 

Our reduction will be from the Label Cover problem. Label Cover combinatorially captures the expressive 
power of a 2-prover 1-round proof system for the problem Max-3SAT(5). Specifically, there exists a reduction 
from Max-3SAT(5) to Label Cover, so that using the well known parallel repetition technique for the specified 
proof system yields a new fc-fold Label Cover instance. For simplicity, we prefer to state our reduction from 
Label Cover and for the sake of completeness, we provide a canonical reduction from Max-3SAT(5) to Label 
Cover. 

Max-3SAT(5) is defined as follows: Given a set of 5n/3 variables and n clauses in conjunctive normal 
form where each clause contains three distinct variables and each variable appears in exactly five clauses, 
find an assignment of variables such that it maximizes the fraction of satisfied clauses. The following result 
is well known jl] [2] : 

Theorem 2.1. There is a constant e > 0, such that it is NP-hard to distinguish between the instances of 
Max-3SAT(5) having optimal value 1 and optimal value at most (1 — e). 

Although this result was proved for general 3CNF formulas, without the requirement that each variable 
appears exactly 5 times, there is a standard reduction from Max-3SAT to Max-3SAT(5) [Hj, which only 
results in a difference in the constant e. 

A Label Cover instance L is defined as follows: 

L={G{V,W,E),{^v,^w),Tl) 

where 

• G{V, W, E) is a regular bipartite graph with vertex sets V and W, and the edge set E. 

• and Yiw are the label sets associated with V and VF, respectively. 

• n is the collection of constraints on the edge set, where the constraint on an edge e is defined as a 
function He : Sy — )■ Sn/. 

A labeling is an assignment to the vertices of the graph, a : {V — ?> Ey} U {W — ?> Sw}. It is said to satisfy 
an edge e = {v,w) if ne(cr(w)) = a{w). The Label Cover problem asks for an assignment a such that the 
fraction of the satisfied edges is maximum. 

A standard reduction from Max-3SAT(5) to Label Cover reveals that 

Theorem 2.2. There is a constant e' > 0, such that it is NP-hard to distinguish between the instances of 
Label Cover having optimal value 1 and optimal value at most (1 — e'). 

In order to amplify the gap, one can define a new Label Cover instance for which the vertex set is 
essentially a set Cartesian product of the original one. This instance, as follows, captures a standard 2-prover 
1-round protocol with parallel repetition £ times applied. We first note that for a given set S = {si, . . . , s„}, 

consists of all ^-tuples of the form (s^^ , . . . , J where Si^ G S and ij runs over {1, ■ ■ ■ ,n} ioi £ > j > 1. 
Given the original Label Cover instance L — (G(V, W, E), (Sy, S^y), 11) reduced from Max-3SAT(5), let 
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where , W^, Ey and are the £ times Cartesian products of the sets V, W, Sy and T,w, respectively 
as defined above. Let 

• consist of all edges of the form e = {v, w) where v = (w^^ , . . . ,Vig) and w = {wi^ , ■ ■ ■ , Wi^) satisfying 
(vi- ,Wi.) & E and for all i> j > I. 

• be the collection of constraints on the edge set E^. The constraint on an edge e = {v,w) where 
V — {vi-^ , . . . ,Vii,) and w — {wi^ , • • • , Wi, ) is a function Ilg : Ey — > which is essentially an ^-tuple 
constraint (Ilg^ , . . . IlgJ, where Ilg^. = !!(„. . .m - .) for > j > 1. 

A labeling a of the vertices V'^ and is said to satisfy an edge e = {v,w) where v = {vi^, . . . ,Vi^) and 
w = (wij , ■ • ■ , Wii^), if ng(cr(w)) — a{w). Note that this requirement is equal to !!(„. •^{a{vi- )) — (j{wi- ) for 

all £ > j > 1. It is easy to see that, in this new Label Cover instance, \V\ = (5n/3)^, \W\ — n^, \E\ — (5n)^, 
|Ey| = 7^ and |E^| = 2^; the degrees of the vertices in V and W is 3^ and 5^, respectively. The following 
theorem is a well known result by Raz |25| : 

Theorem 2.3. There is an absolute constant a > 0, such that it is NP-hard to distinguish between the case 
that OPT{L^) = 1 and OPT{L^) < 2""^. 

3 Exponential Inapproximability of MAX- VOL 

3.1 The Basic Gadget 

At the heart of our analysis is a set of vectors with a special property. We will use a set of vectors (composed 
of binary entries for simplicity of construction) such that any two of them have large dot-product. We will 
also require that the dot product of a vector and the binary complement of any other vector is large. More 
specifically, we need these dot products be proportional to the Euclidean norms squared of the vectors. 

Given a vector v = (vi . . . Vm) where Vi G {0, 1} for m > i > 1, we denote the binary complement of v by 
V = (zJT . . .uin) where vl = \ ii vt — Q, and vl = Q otherwise. We begin with the following lemma: 

Lemma 3.1. For m > 2, there exists a set of vectors B — . . . , 62™-i} of dimension 2™ with binary 
entries such that the following three conditions hold: 

1. ||6,||2 = 2(™-i'/2 for2m-l>i>l 

2. b,-b]^ 2'"-2 for2m-l>i>j>l. 

3. h ■ bj = 2™-2 for2m-l>i>j>l. 

Proof. Consider the Hadamard matrix H of dimension 2™ x 2™ with entries —1 and 1, constructed recursively 
by Sylvester's method. Let B be the (2™ — 1) x 2"* matrix consisting of the rows of H for which we replace 
— I's with O's, excluding the all I's row. We claim that the rows of B satisfy the requirements. Indeed, by the 
properties of Hadamard matrices, each row of B has exactly 2™^^ I's which satisfies the first requirement. 
Note also that, for m > 2, two distinct rows of H (excluding the all I's vector) have exactly 2™~^ element- 
wise dot-products of the following four types: 1-1, 1 • (— 1), (— 1) • 1, (—1) •(—!). Considering the construction 
of B, we have that the dot-product of any two of its rows is 2™^^ since all the products in H involving — 1 
vanishes for B. Similarly the dot-product of a row with the binary complement of another row is 2™^^ by 
symmetry. Thus, the second and the third requirement also hold. □ 
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3.2 The Reduction 



Lemma [3.11 guarantees the existence of of a set of binary vectors B — {bi, ...,62*} of dimension 2^+^ such 
that the following three conditions hold: 

1. \\b,\\^ ^ 2^/2 for 2^ > i > 1 

2. b,-b] = 2^-1 for 2'^ >i> j>l. 

3. 6, • bj = 2'^-^ for 2^ > i > j > 1. 

B can be constructed in time 0(2^^). In our reduction, £ will be a constant (to be exactly determined later) 
inversely proportional to a which is the constant in Raz' Theorem. Hence, one can construct B in constant 
time. For the sake of simplicity of our argument, we normalize the vectors in B, which then clearly satisfies 

1. ||6,||2 = 1 for 2*" > i > 1 

2. b,-b] =1/2 for 2^ > i > j > 1. 

3. b, ■ bj = 1/2 for 2^ >i> j >1. 

Given a Max-3SAT(5) instance and the reduction described in the previous section, we will define a 
column vector for each vertex-label pair in L^, making (35n/3)^ + (2n)^ vectors in total. (Note that \V^\ = 
(5n/3)^ \W'^\ = = {1, . . . , 7^} and Sf^ = {1, . . . 2^}). Each vector will be composed of \E^\ = (5n)^ 

"blocks" which are either vectors from the set B or the zero vector according to the adjacency information. 
More specifically, let Ay^i be the vector for the vertex label pair v £ and i G Sy. Similarly let A^ij be 
the vector for the pair w G W and j S S^. Both of these vectors are (5n)^2^+^ dimensional. The block 
of Ay_i corresponding to an edge e G is denoted by Ay^i{e). The block of Ay^j corresponding to an edge 
e £ is denoted by Ay,j{e). We define 



In order to show how our reduction works, we present a part of a simple bipartite graph in Figure [ijwith all 
the edges drawn between two pairs of nodes, and the corresponding (row) vectors computed by the reduction 
in Figure [5] Note that Ay^i has exactly 3^ non-zero blocks, and Ayjj has 5^ non-zero blocks. Hence, according 
to the definition above, their Euclidean norm is 1. The column vector set for the MAX- VOL instance is 
defined as 



Note that M = (5n)^2^+i and N = (35n/3)^ -I- (2n)^ both having polynomial size in n for constant £. From 
an intuitive point of view, we define mutually orthogonal subspaces for each edge, and then we "spread" 
the Euclidean norm of each vector to the subspaces corresponding to the edges incident to the vertex 
corresponding to the vector. A crucial observation for this construction is that, vectors ^t>i,ii and Ay^^i^ 
are orthogonal to each other for all vi,V2 G V^, and 11,12 S Sy, since there are no edges between the 
vertices in V^. The same result holds for the vertices in W^. From now on, this fact will be used frequently 
without explicit reference. We set the number of column vectors k to be chosen in the MAX- VOL instance 
to + \ W^\ = {bn/?>Y + ■ Note that fc is a constant fraction of N , the total number of columns, i.e. 
there exists a constant 5 <1 such that k = SN. 
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3.3 Analysis 



We start with the completeness of the reduction: 

Theorem 3.2. If the Label Cover instance has a labeling that satisfies all the edges, then in the MAX-VOL 
instance, there exist k column vectors with volume 1. 

Proof. We show that there are at least k orthogonal vectors. For an edge e = {v, w), let i € and j € 

be the labeling of v and w assigned by the optimal labeling which satisfies all the edges. Then, in the 

MAX-VOL instance the dot product of the vectors A^^i and A^^j is 

Av,i ■ A^j = ^ A„^j(e) • A^j{e) = 6nf(i) • bj = bj ■ bj = 0. (2) 

This is due to the fact that the labeling satisfies e, i.e. 6n*(i) — bj. Since all the edges are satisfied, and 
there exists a vector from each vertex corresponding to the optimal labeling satisfying the equation ([2]), we 
have + \ W^\ orthogonal vectors, i.e. we have k orthogonal vectors. □ 

Before proving the soundness of the reduction, which will prove hardness of approximation, we first give 
the intuition for the argument. According to our construction of the MAX-VOL instance, there is a set of 
vectors corresponding to each node in and W^. The set of vectors defined for a specific node has high 
pair-wise dot products whereas a vector from a node vi G and another from V2 in are orthogonal to 
each other. The same goes for the vectors defined for W^. Hence, if vectors are chosen from the same set 
corresponding to a single node, the total volume will decrease exponentially with respect to the number of 
such vectors. Let us call these vectors duplicates in and W^. The more intricate part of the analysis is 
due to the dot products between the vectors defined for and W^, which is enforced to be non-zero by 
the unsatisfied edges in the Label-Cover instance. We will show that, in case the Label-Cover instance has 
few satisfied edges, any k vectors chosen in the MAX-VOL instance should satisfy the following: either the 
number of duplicates in and is large enough so that the total volume is small, or the dot products 
between and leads to a small volume. 

Theorem 3.3. There exist absolute constants a and c such that, if the Label Cover instance L^ does not have 
any labeling that satisfies more than 2~"^ of the edges, then the volume of any k vectors in the MAX-VOL 
instance is at most 2^'^^ . 

Proof. Let — {wi, . . . , U(5„/3-)«} and = {wi, Let A^ be the vectors corresponding to the 
vertex v S V^: Ay = {Ay^i\i S Ey}. Similarly, let A^j = {Ayjj\j e E^} for w € W^. Let Ayi be the set of 
all vectors corresponding to the nodes in V^, and Ayyi be the set of all vectors corresponding to the nodes 
in W^, i.e. 




Figure 1: A part of a simple bipartite graph representing a Label-Cover instance 
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Figure 2: The resulting (row) vectors in MAX- VOL instance computed from the graph in Figure [T] by our 
reduction 



(5«/3)' „^ 

Ayi =^ y A^,, A^yi^IjA^,. 

1=1 i=l 

For a set of vectors C of size fc, let Cu = CDAu for all u G {V^UW'^}, Cyi = CnAyt and Cw^ = C^]A^rt. 
Let y^(C) and W^{C) be the set of vectors for which C "selects" at least one vector from and W^, 
respectively. 

V'{C) = {ve V'\C, ^ 0}, W'{C) ^{we W'\C{A^) ^ 0}. 

For ease of notation, we let ky^ = \Cyi\,kwc = I^'h"! |, dye = ^Vc ~ l^^(C)|,c?VKc — ^Wc ~ Note 
that ky^ and k\Yc denote how many vectors are chosen by C from and W^, respectively. Whereas dy^. 
and dwc ^-re the total number of duplicates in Cye and , respectively. The following lemma relates the 
number of duplicates on one side with its volume. 

Lemma 3.4. Vol{Cvi) < (V3/2)''^c and Vol{Cw'-) < (\/3/2)^"'c . 

Proof. Let P be the set of |y^(C)| elements which contains exactly one vector of the form A^^i for each 
V £ V^{C). In words, we consider the vectors of C corresponding to the nodes in the Label-Cover instance 
minus all the duplicates. For the duplicate vector A^^j, we have A^^i ■ Ayj — 1/2. Hence, d{Ayj,P) < 
d{Ayj,Ay^i) = \/3/2. By the definition of dy^ and by the Union Lemma, we get Vol{Cye) < (•\/3/2)'^^c . 
The argument for Vol{Cy^^e) is similar. □ 

Let the constant c — 1/(3 • 5^^^). Recall that, our reduction will require £ to be inversely proportional to 
a in Raz' Theorem. Hence, although having an exponential dependence on a, c is a constant. We will show 
that Theorem 13.31 holds for this value of c; we will prove that Vol{C) < 2~^'^ for any set C of k vectors. To 
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this aim, we argue by contradiction. The next lemma roughly states that if the volume of C is large enough, 
then its vectors are almost equally distributed among the nodes of the Label-Cover instance. This condition 
will in turn imply a small volume completing our argument. 

Claim 3.5. // Vol{C) > 2-'='' for c = 1/(3 • 5^+^), then 

(1 - ei)(5n/3)^ < kva < (1 + ei)(5n/3)^ (3) 

(1 - e2)n^ < kwc < (1 + <^2W, (4) 
where ei = ^ ((3/5)^ + (3/5)2^) and 62^^ ((3/5)^ + l). 

Proof. First, we note that Vol{C) < Vol{Cyi) since all the vectors in the MAX- VOL instance have unit 
norm. Similarly, Vol{C) < Vol{Cw>:)- Thus, by the premise of the claim, we have Vol{Cvi) > 2^"^^ and 
VoliCwi) > 2-"^. By Lemma [311 we get 



{Vi/2f^c = 2''^c(-i+i°g3/2) > Vol{Cvi) > 2" 



■ck 



which implies dvc < cfc/(l — log 3/2) < 5cfc since log 3 < 1.6. The analysis for dwc along exactly the same 
lines also yields dwc < 5cfc. Noting the expressions for c and fc, and following the definitions, we obtain 



= \V\C)\ + dvc < IV^I + 5ck 
= (5n/3)^ + 
= (l + ei)(5n/3r. 



{5n/3Y + ^{{5n/3Y + n') 



Similarly, 



kwc = (C)| + dwc < I + 5cfc 

= (1+62^ 

which proves the right hand sides of ^ and Noting that kvc + kwc — k — (5n/3)^ -I- n^, we get 

kvc = k- kwc > {5n/3Y + - (1 + 62)^^ 

= i5n/3Y-^ii3n/5Y + n') 
= (l-ei)(5n/3/ 

and 

kwc =k- kvc > {^n/iY + - (1 + ei)(5n/3)^ 

= "'-^((3^/5)' + "') 
= (1 - e2)n' 

which proves the left hand sides. □ 
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Claim ensures that if the volume of a set of k vectors exceeds 2"'^'^, then some certain concentration 
result should hold, namely Equation ([3]) and Equation Q. We will now show that, these equations imply 
Vol{C) < 2~'^^ which is our contradiction. 

Without loss of generality, let V'^{C) — {ui, . . . , Vq}, VF^(C) = {wi, . . . , Wp}. Note that these sets contain 
the nodes of the Label-Cover instance from which C "selects" at least one vector. Let Q = {Ay^,i^ , ■ ■ ■ , ^i>,,i,} 
where ^u^.i, G Cy^ for s = 1, . . . , q. Let P = {Aujui, . . . ,^iupjp} where Ay^^i^ e for s = 1, . . . ,p. By 
definition, 

q = kvc ~ dvc > (1 ~ 2ei)(5n/3)^, p = kwc — dwc > (1 ^ 2e2)n^. 

In words, the set of nodes from which C selects at least one vector essentially covers and . These 
vectors are all orthogonal. From this point of view, V^{C)) and W^^(C)) play an important role in our 
argument. Since C "covers" and and since the Label-Cover instance has many unsatisfied edges, 
it means that the dot products of many vectors in Cye with many vectors in C^n will be large. This 
will lead to small volume. Hence, we are essentially interested in the number of unsatisfied edges between 
V'^{C) and W^{C). Since there are at most 2~"^ satisfied edges in the Label-Cover instance, and there 
are exactly 3^ edges incident to a node in , the number of unsatisfied edges incident to V^{C) is greater 
than (1 — 2ei — 2~"^)(5n)^. Similarly, the number of unsatisfied edges incident to W^{C) is greater than 
(1 — 2e2 — 2^"^)(5n)^. Thus, the number of unsatisfied edges whose end points are in V^{C) and W^{C), is 
greater than (1 - 2ei - 2e2 - 2-"^+i)(5n)^. 

We now give an upper bound for the distance of the vectors in Q to P, namely 11^^,^^^^ — TTp{Ay^_i^)\\^ for 
each Ay_,^i^ e Q. To this end, we define the set N{Ay^^i^) = {Cu,|e = {Ay^^i^^w) is unsatisfied}. Note that 
the vectors in different sets are mutually orthogonal, and by the reduction we have 

Avs,is ■ ^w,j = ^ Ay^^i^{e) ■ Ay,^j{e) — fenf(i) • = 2 • 3^/2 . 5^/2 
for Ay,j e N{Ay^^i^) since e = {Ay^^i^,Ay,j) is unsatisfied. Thus, by the Pythagoras Theorem, we obtain 

d{Ay^,^,P) = \\Ay^,^ - -^PiAy^.M^ < (1 - ' ■ 

Using the Union Lemma, we get 

Vol{P\JQ) < Vol{P) ■\ld{Ay^^,^,P) 

<-(-)-n'(-^)*- 

s—l ^ ^ 



The product in the last expression is maximized when all the factors are equal to each other. We also 
previously showed that Y11=i [-^(^fa.is)! > (1 ~ 2ei — 2e2 — 2~"^+-'^)(5n)^ and that (7, the number of distinct 
nodes hit in satisfies, q> {1 — 2ei)(5n/3)^. Hence, we obtain 
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Vol{P U 0) < VoliP) . n (l - ^f^i^^^) ' 

< Vol{P) ■ \{ (I (5^/3). .4. 3^ .5^ ) 

(l-2ei)(5r./3)'' 

/ d _ 2ei - 2e2 - 2""^+^' 

< Vol{P) • 1 - ^ ; 



4-5^ 

To simplify, let t = (^_2e^_2e2-2-"^+i) • ^ — 1' have 

ei = ^ ((3/5)^ + (3/5)^0 < ^ ((3/5) + (3/5)^) < 3/20. 
Noting that loge > 10/7, we obtain loge • (1 - 2ei) > 10/7 • 7/10 = 1. Then, we get 

^ (l-2ei)(5r./3)<' 

Vol{P UQ)< VoliP) • ( 1 " 7 ) 

< e 2t 

, (l-2ei ){5?i/3)* 

(5r./3)^ 

< 2 ~, 

where e is the base of the natural logarithm. In the second inequality, we have used the fact that Vol{P) < 1 
and (1 - i)* < e-i for t > 1. 

We will now provide an upper bound for t to further simplify the last expression. To this aim, let i' be 
the smallest integer such that 2~"^ +^ < 11/27. Taking logarithms and rearranging, it is easy to see that 

e = . Note also that for ^ > 2, we have 2ei < 1/27 and 2e2 < 3/27. Then, for i = i', we get 

_ 4-5^ 4-5^ _ 4 • 5^ _ ^ 

*-(l-2ei-2e2-2-^+i) <(l-^-A_n)-(i7^-9-5. 

Since k = (5n/3)^ + n^, we also have 

= fc/(l + (5/3)0 > k/{5/3Y+\ 



which yields 



(5n/3)' k , 

Vol{P U Q) < 2 = 2"i^+^ < 2"Fi^+T = 2-"=*=, 



which is our contradiction. Thus, the volume of a set of k vectors in a negative instance of MAX- VOL cannot 
exceed 2^''^ for c = ■ □ 

We have shown that 

• if the optimal value of the Label Cover instance is 1, then the optimal value of the MAX- VOL instance 
is 1. 
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• if the optimal value of the £-fold Label Cover instance is less than 2 " , then the optimal value of the 
MAX- VOL instance is less than 2"°'=. 

By the combination of Theorem 12.11 and Theorem I2.3[ we know that there exists a gap producing reduction 
from SAT to £-fold Label Cover with parameters 1 and 2~"^. This means that there is a polynomial time 
reduction from SAT to MAX- VOL such that, given a formula (p 

• if is satisfiable , then OPT(MAX-VOL) = 1. 

• if is not satisfiable , then OPT(MAX-VOL) < 2-"''. 

Thus, unless P = NP, MAX- VOL is inapproximable within 2^^'^ for some constant c > 0. 

4 Discussion 

Our reduction heavily relies on the Raz' Parallel Repetition Theorem 125'. Indeed, it doesn't seem possible to 
get an exponential inapproximability result without parallel repetition. But, since the degrees of the vertices 
in the Label-Cover instance exponentially increases with respect to the number of repetitions, our constant 
c depends on the constant a in Raz' result. It might be possible to improve this constant by making use of 
more sophisticated parallel repetition theorems, but we did not proceed so far. Indeed, the exact analysis 
is irrelevant as the constant will be too small in all cases. Overall, the strength of our result is is directly 
related to the underlying theorems for the inapproximability of Label-Cover. 

Another way of getting a stronger hardness result is to find a more sophisticated reduction. In our MAX- 
VOL instance, the subspaces "reserved" for each edge in the Label-Cover instance are orthogonal to each 
other. This dramatically simplifies the analysis, yielding perfect completeness, i.e. volume 1 in MAX- VOL. 
It might be possible to construct a MAX- VOL instance for which these subspaces have some pair-wise angle, 
so that we sacrifice the perfect completeness, but at the same time get a much smaller soundness. This 
would improve the inapproximability result. 

The obvious open problem is whether the inapproximability can be strengthened to 2^^^^^. Recall that 
this is the lower bound for the greedy algorithm for MAX- VOL. Considering the multiplicative nature 
of the problem yielding a very small approximation ratio for the obvious greedy algorithm, a significant 
improvement of the upper bound would be expected to provide asymptotically better approximations in the 
exponent. This suggests that the inherent hardness of MAX- VOL might be very close to the performance of 
the greedy algorithm. However, with the techniques we have used, it is not possible to break the dependence 
of c on the constant in the parallel repetition theorems. 

We would finally like to point out that the reduction and the analysis provided in this paper might be a 
good starting point for studying hardness of other matrix approximation problems in general (e.g. [3j llOj ) 
for which no technique related to the PCP theorem have been used. Such an extension to other matrix 
approximation problems is not trivial. Indeed, computing the volume is already difficult although purely 
geometric intuition is used. Relating this to other linear algebraic functions (e.g. singular values) which 
continuously depend on the entries will put even more strain on the analysis. 

Acknowledgments: We would like to thank loannis Koutis who, in the final stages of this paper, 
pointed out to us the relevant lines of research in V-polytope theory [2T1. 
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